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Response to Amendment 

The amendment received 1/20/06 has been entered in full. 

Response to Arguments 

Applicant's arguments filed 1/20/06 have been fully considered but they are not 
persuasive. Specifically the Applicant argues that Corby, Jr. does not disclose that the 
reference signals are captured using the same imaging device used to capture image 
data of the object, or storing the orientation of the robot with respect to the reference 
object. Examiner respectfully disagrees. As stated in the previous office action it is well 
known to capture images of an environment in known conditions in order to use them as 
reference data for orientation purposes later on. Corby discloses the use of manipulator 
arms typically requires a method of determining the position and orientation of the distal 
end of the manipulator arm (col. 1 lines 44-46). Corby further discloses moving the 
manipulator arm to a particular position and videotaping the structure or device which is 
to be examined (col. 1 lines 56-58). At a later date the manipulator arm is positioned at 
the same site and current data (such as a video image) is compared to previous data 
(col. 1 lines 58-60). One of ordinary skill in the art would find the disclosure of Corby 
enabling. The Applicant further argues that Corby does not disclose that reference 
images are captured. The Examiner respectfully disagrees, the previous data that is 
being compared to the current data is read as the reference image, thus this limitation is 
disclosed. 
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Claim Rejections - 35 USC § 102 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

2. Claim 33 is rejected under 35 U.S.C. 102(e) as being anticipated by Garibotto et 
a!. USPN 5,91 1,767 (hereinafter "Garibotto"). 

Garibotto discloses a robot system (robot navigation system, figure 8) comprising 
a robot (figure 8); an image capturing device capturing image data of a plurality of 
objects (image acquisition means, figure 8, col. 7 lines 10-23); a memory storing 
reference models, each reference model including an image of a reference object 
captured by the image capturing device in a different direction (means for storing data 
regarding environment, figure 8, col. 7 lines 10-23); and a processor to perform 
matching on the image data containing images of the plurality of objects captured by the 
image capturing device with each of the reference models successively to select one 
object having an image matched with one of the reference models (computer, figure 8, 
col. 7 lines 10-23, and col. 7 lines 39-43). 
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Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, rf the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-4, 12-15, 23 and 24 are rejected under under 35 U.S.C. 103(a) as being 
unpatentable over Spight (USPN 4,462,046, previously cited), in view of Corby, Jr. et al. 
(USPN 5,745,387, hereinafter "Corby"). 

5. Regarding claim 1 , Spight discloses a robot system having an image processing 
function for determining orientation, or orientation and position of a robot operation on 
one of a plurality of objects (abstract), the system comprising: a robot 202 (Figure 2); a 
first image capturing device 40 capturing image data of the plurality of objects 
containing respective images of the objects (Figure 1 ; column 3, line 29-38; column 5, 
line 34-48; column 6, line 11-13); Spight further discloses a memory for string reference 
images wherein each desired object to be identified would be defined by a plurality of 
reference signals indicative of a number of particular orientations (capturing direction) of 
the object being viewed (col. 9 lines 4-19, col. 3 lines 29-38). Spight further discloses a 
processor 64 to perform matching (correlation) on the image data containing images of 
the plurality of objects captured by said first image capturing device with each of said 
reference models successively to select one object having an image matched with one 
of said reference models (column 7, line 64-column 8, line 15; column 9, line 59-column 
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10, line 11), and to determine orientation, or orientation and position of the robot 
operation based on the image of the selected one object, based on said one reference 
model and the information of its associated capturing direction (column 7, line 28-49; 
column 8, line 16-37; column 1 1 , line 21-37, it is necessary that the orientation of the 
robot must be determined based on the orientation of the object in order to drive the 
robotic effector). 

Spight does not explicitly disclose that the reference signals are captured using 
the same imaging device used to capture image data of the object, or storing the 
orientation of the robot with respect to the reference object. However it is well known to 
capture images of an environment in known conditions in order to use them as 
reference data for orientation purposes later on. Corby discloses that the use of 
manipulator arms typically requires a method of determining the position and orientation 
of the distal end of the manipulator arm (and/or subparts) (col. 1 lines 44-46). Corby 
further discloses moving the manipulator arm to a particular position and videotaping 
the structure or device which is to be examined (col. 1 lines 56-58). At a later date the 
manipulator arm is positioned at the same site and current data (such as a video image) 
is compared to previous data (col. 1 lines 58-60). 

Spight and Corby are analogous art because they are from the same field of 
endeavor of using image processing to manipulate objects in an environment using a 
robot. 

At the time of the invention it would have been obvious to one of ordinary skill in 
the art to combine Spight and Corby to obtain the reference images using the same 
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imaging device . Spight discloses that the images of multiple orientations of the objects 
are compared with reference objects, and Corby discloses the conventional method of 
producing reference objects. The motivation for using the same camera to take the 
reference image and the later images is that it would allow the manipulator arms to 
determine physical changes in the environment. Thus it would have been obvious to 
one of ordinary skill in the art to combine Corby and Spight to obtain the invention as 
specified in claim 1. 

Claim 12 is similar to claim 1, thus the same arguments used for claim 1 apply 
equally to claim 12. 

Regarding claims 2 and 13, Spight discloses that said reference models are 
obtained from a part of the image data of the reference object (column 3, line 29-38; 
column 6, line 37-52; column 10, line 50-63). 

Regarding claims 3 and 14, Spight discloses that said reference models are 
obtained by processing the image data of the reference object (column 3, line 29-38; 
column 6, line 37-52; column 10, line 50-63). 

Regarding claims 4 and 15, Spight discloses that said first image capturing 
device 40 comprises a camera for capturing two-dimensional image data (column 5, line 
42-48). 

Claim 23 is similar to claim 1 , thus the same arguments used for claim 1 apply 
equally to claim 23. 

Regarding claim 24, Spight discloses that reference images and reference 
arrangement information is obtained for workpieces/objects of different shapes (plurality 
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of configurations of each desired object to be identified), and wherein the finding 
comprises first determining that a reference image of one of the different shapes 
matches the working image of the workpiece, and then finding one reference image of 
the shape that best matches the working image (column 8, line 10-30; column 9, line 4- 
19). 

6. Claims 5 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Spight, in view of Corby, Jr. et al., and further in view of Suyama et al. (USPN 
4,879,664, previously cited, hereafter Suyama) or Stauffer (USPN 4,410,804, previously 
cited). 

Spight nor Corby disclose that the image data are captured from a predetermined 
distance. Suyama discloses a three-dimensional position sensor comprising robot- 
teaching apparatus wherein said image data of the reference object are captured by 
said camera (Figure 11a, element 35) from a predetermined distance (column 6, line 
20-37). In addition, Stauffer teaches that if a two-dimensional sensor is used, the image 
processor is unable to simultaneously determine the distance to the object unless the 
objects are always positioned at a known distance (column 1 , line 31-39). It would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
capture an image at a predetermined distance in order to adjust the position and 
posture of teaching head 3 with respect to the workpiece 9 as taught by Suyama 
(column 6, line 20-37) or such that the distance to the object need not be determined by 
other means as taught by Stauffer (column 1 , line 31-39). 
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7. Claims 6, 7, 11, 17, 18 and 22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Spight, in view of Corby, Jr. et al., and further in view of Maeno et al. 
(USPN 5,047,714, previously cited, hereafter Maeno). 

Regarding claims 6 and 17, Spight nor Corby disclose that the robot moves a 
second image capture device to have the determined orientation and/or position. 
Maeno discloses a method of recognizing surface-mounted parts including a robot 
system comprising a second image capturing device (Figure 7c, element 10); wherein 
said robot situates said second image data capturing device to have said determined 
orientation or to have said determined orientation and said determined position with 
respect to the selected one object (column 4, line 44-49), and a processor processes 
second image data captured by said second image capturing device to detect position 
and/or rotational posture of the selected one object with respect to said second image 
data capturing device (column 4, line 54-63). It would have been obvious to one of 
ordinary skill in the art at the time the invention was made to move the second image 
capture device (imaging tube) to have the determined orientation and/or position as 
taught by Maeno in order to locate position detecting patterns located at specific 
positions to accurately and directly determine the position and direction of conductor 
patterns on the printed circuit board (column 1, line 51-59). 

Regarding claims 7 and 18, Spight nor Corby disclose that the robot moves a 
second image capture device to have the determined orientation and/or position. 
Maeno discloses a robot system comprising a second image capturing device for 
obtaining three-dimensional position; wherein said robot situates said second image 
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data capturing device to have said determined orientation or to have said determined 
orientation and said determined position with respect to the selected one object, so that 
the second image data capturing device is directed to a characterizing portion of the 
object; and wherein said processor detects the three-dimensional position and/or 
posture of the selected one object based on the position of said characterizing portion 
obtained by said second image data capturing device (see discussion of claims 6 and 
1 7 above). Maeno does not explicitly disclose that the three-dimensional position of the 
selected one object, but the examiner takes Official Notice that obtaining three- 
dimensional position of an object is well known in the art. It would have been obvious to 
one of ordinary skill in the art at the time the invention was made to detect the three- 
dimensional position of the object in order to determine the location of the object in 
space so that operations may be carried out upon it. 

Regarding claims 1 1 and 22, Spight discloses that said robot operation is an 
operation of picking up the selected one object from the plurality of objects, some of 
which are overlapped (randomly arranged) with each other (column 6, line 11-13). 

8. Claims 8, 19, 27 and 30 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Spight, in view of Corby, Jr. et al., in view of Maeno and further in 
view of Soderberg (USPN 4,785,528, previously cited). 

Regarding claims 8, 19, 27, and 30, Maeno does not disclose that said first 
image data capturing device is used as said second image data capturing device. 
Soderberg discloses a robotic work positioning system including a camera 34 mounted 
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on member 28 of robot 10 which serves as both the first and second image data 
capturing device since it both determines the position of the work, and is moved to have 
the determined orientation and/or position (Figure 1; column 2, line 53-column 3, line 1, 
line 44-50; column 5, line 34-41 ). It would have been obvious to one of ordinary skill in 
the art at the time the invention was made to use said first image data capturing device 
as said second image data capturing device as taught by Soderberg in order to retrieve 
parts from a conveyor using a vision system to select the desired part from other parts 
and engage the part at precisely the correct location to be properly placed by the robot 
(column 5, line 60-column 6, line 6). 

9. Claims 25 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Spight, in view of Corby, Jr. et al., and further in view of Soderberg. 

Regarding claim 25, Spight nor Corby disclose that the robot is used to capture 
the reference images. Soderberg discloses a robot that is used to capture reference 
images, and wherein the reference arrangement information represents arrangements 
of the robot when capturing the reference images (Figure 1 ; column 2, line 53-column 3, 
line 4, line 31-59; column 5, line 60-column 6, line 6). It would have been obvious to 
one of ordinary skill in the art at the time the invention was made to use a robot to 
capture reference images as taught by Soderberg in order to retrieve parts from a 
conveyor using a vision system to select the desired part from other parts and engage 
the part at precisely the correct location to be properly placed by the robot (column 5, 
line 60-column 6, line 6). 
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Regarding claim 26, Spight nor Corby disclose that a second imaging device is 
affixed to the robot. Soderberg discloses a camera 34 mounted on member 28 of robot 
10 and is used to determine additional arrangement information used to determine the 
known arrangement of the robot relative to the workpiece (Figure 1 ; column 3, line 44- 
50; column 5, line 34-41 ). It would have been obvious to one of ordinary skill in the art 
at the time the invention was made to affix an imaging device to the arm of a robot as 
taught by Soderberg in order to retrieve parts from a conveyor using a vision system to 
select the desired part from other parts and engage the part at precisely the correct 
location to be properly placed by the robot (column 5, line 60-column 6, line 6). 

10. Claims 9, 10, 20, 21, 28, 29, 31 and 32 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Spight, in view of Corby, Jr. et al M in view of Maeno, and 
further in view of Sakakibara et al. (JP 07-270137, previously cited, hereafter 
Sakakibara). 

Regarding claims 9, 20, 28, and 31 , Maeno does not disclose that said second 
image capturing device comprises a three dimensional visual sensor of spot-light 
scanning type. Sakakibara discloses a three dimensional visual sensor usable in robot 
automation (page 1, paragraph 1), of spot-light scanning type capable of measuring 
distance between the sensor and an object (page 3, paragraph 15). It would have been 
obvious to one of ordinary skill in the art at the time the invention was made to utilize a 
three dimensional visual sensor of spot-light scanning type as taught by Sakakibara in 
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order to quickly and accurately determine the three dimensional position of an object 
using one device (page 2, paragraph 11). 

Regarding claims 10, 21, 29, and 32, Sakakibara discloses an image data 
capturing device comprising a structured-light unit for irradiating a structured light on the 
selected object and capturing an image of the object including the irradiated light on the 
object (page 3, paragraph 16). 

Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX -MONTHS from the date of this final action. 

Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to John B. Strege whose telephone number is (571 ) 272- 
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7457. The examiner can normally be reached on Monday-Friday between the hours of 



If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on (571) 272-7453. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-21 7-91 97 (toll-free). 
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